Field of the Invention
The present invention relates generally to videoconferencing and more specifically to a telepresence system with specific features such as automatic matching the height of the telepresence device's display to a local subject, maintaining a selected distance with the local subject, recognising gaze direction and matching eye contact between a remote user and the local subject.
Description of Related Art
Driven by globalization of business organizations and decentralization of workforce, demands of video conferencing services and equipment are in rapid expansion in multinational corporate enterprises. In recent years, video conferencing has become widely adopted by public sectors and healthcare sectors as an effective form of communication between remote users. However, messages delivered by video conferencing is not as effectively as it is in a face-to-face conversation, particularly in a sense that subtle non-verbal communication cues such as eye contact, eye gaze and interpersonal distance, are often overlooked or impossible to be replicated in video conferencing. By contrast, people engaged in a face-to-face conversation tend to frequently keep an eye contact with others within an interpersonal distance.
In most video conferencing systems and telepresence systems, cameras or image sensors, which capture an image of the interacting subject, are often located above or at the periphery of the monitor, which displays an image of the remote user. It is evident that even a user is intended to look into the eyes of an interacting subject's image in the monitor will appeared to be looking down in the interacting subject's perspective when the camera is placed atop the monitor. Similarly, the user will appear to be gazing to a side when the camera is placed on the side of the monitor, or will appear to be looking up when the camera is placed below the monitor. As a result of this camera orientation issue, the user's gaze direction shown in the subject's monitor will appeared different from his actual intention. However, adjustments on the images are often neglected in these systems. These systems, therefore, are insufficient for preserving gaze direction or maintaining eye contact between users.
Telepresence robotic system remains one of the most realistic two-way interactive video conferencing available on the market hitherto because it overcomes physical distance while assists to create a sense of presence of a remote user to enable social communication. Advancements have been made to improve the interaction between users, particularly, to permit eye contact between the user and the interacting subject via a telepresence device. Proposed methodologies in these advancements include manoeuvring the head of the telepresence device to face the direction of a user's voice, and using a semi-silvered mirror or a beamsplitter panel that allows mounting a camera directly behind to provide an apparent eye contact between the user's image and the interactive subject. Another teaching reveals a telepresence system that relies on a projected life size image of the surrogate's location to preserve the user perspective and a vertical gaze. Another teaching uses 3D reconstruction technique to re-center both the head and the eye position of the user's image in the telepresence device display.
However, these telepresence systems have their limitations in terms of ease of use, affordability and overall user experience. For example, there is a need of heavy computation and networking resources to render and transmit the high resolution life size background image, the user's intended gaze direction is not truly preserved and there are inherent issues of beamsplitter, such as degraded image quality by ambient light and bulkiness of the setup that restricts the mobility of a user to a specific physical location where the beamsplitter is installed beforehand.
More recently, a telepresence robotic (E-Travel) system has been developed that duplicates the height of a remote user in a surrogate robot and preserves the remote user's perspective and vertical gaze. In another approach, a telepresence device equipped with haptic feedback is configured to move in response to a force exerted by a local subject. However, none of these systems show capability of automatically matching the robot's height with the local subject or continuously maintaining a selected distance between the robot and the local subject in response to the real-time movement of the subject.
Behaviour scientists have shown that having a commonly accepted interpersonal distance according to various circumstances and maintaining constant eye contact have major impacts on social interaction. A lack of these interactions may create a perception of disinterest or preoccupation that made one feel disconnected from the conversation, or may lower the attention spans among the participants and may made difficult for the remote user to maintain a sense of presence at the same physical location as the interacting subject. Furthermore, researches have shown that person who is video conferencing and is presented as eye contact avoidant, due to the camera parallax error, will be perceived negatively. Results have also shown that people prefer interacting with devices of their similar height and tend to be staying closer to the telepresence device when they are about the same height.
Accordingly, in order to have a better sense of personal contact and to reinforce the presence of the remote user, there is a need in the art for a method or a system that specifically preserve the intended eye gaze of both users while capable of responding to the physical movement of the subject in real-time, such that the telepresence device automatically maintains a selected distance with the local subject and matches with the subject's height.